Reuse and Challenges in Evaluating Language Generation Systems: Position Paper
نویسنده
چکیده
Although there is an increasing shift towards evaluating Natural Language Generation (NLG) systems, there are still many NLG-specific open issues that hinder effective comparative and quantitative evaluation in this field. The paper starts off by describing a task-based, i.e., black-box evaluation of a hypertext NLG system. Then we examine the problem of glass-box, i.e., module specific, evaluation in language generation, with focus on evaluating machine learning methods for text planning.
منابع مشابه
Automatic Evaluation of Referring Expression Generation Is Possible
Shared evaluation metrics and tasks are now well established in many fields of Natural Language Processing. However, the Natural Language Generation (NLG) community is still lacking common methods for assessing and comparing the quality of systems. A number of issues that complicate automatic evaluation of NLG systems have been discussed in the literature. 1 The most fundamental observation in ...
متن کاملA Component-Oriented Programming for Embedded Mobile Robot Software
Applying software reuse to many Embedded Real-Time (ERT) systems poses significant challenges to industrial software processes due to the resource-constrained and real-time requirements of the systems. Autonomous Mobile Robot (AMR) system is a class of ERT systems, hence, inherits the challenge of applying software reuse in general ERT systems. Furthermore, software reuse in AMR systems is chal...
متن کاملReliability Assessment of Distribution Systems in Presence of Microgrids Considering Uncertainty in Generation and Load Demand
The microgrid concept provides attractive solutions for reliability enhancement of power distribution systems. Normally, microgrids contain renewable-energy-based Distributed Generation (DG) units, which their output power varies with different environmental conditions. In addition, load demand usually changes with factors such as hourly and seasonal customer activities. Hence, these issues hav...
متن کاملSoLDES: Service-oriented Lexical Database Exploitation System
In this work, we focuses on the assisted exploitation of lexical databases designed according to the LMF standard (Lexical Markup Framework) ISO-24613. The proposed system is a service-oriented solution which relies on a requirement-based lexical web service generation approach that expedites the task of engineers when developing NLP (Natural Language Processing) systems. Using this approach, t...
متن کاملEvaluating Challenges of the National Cryptocurrency
Today, cryptocurrencies in global payment systems have been proposed as a way to become independent of traditional banking and to get out of the control of banks and monetary policies of governments and reduce fraud in banking transactions and counterfeit them. In this paper, we create a comprehensive picture which includes the challenges of this field, and we analyze the results with a case st...
متن کامل